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Abstract 

Wireless systems comprised of rechargeable nodes have a significantly prolonged lifetime and are 
sustainable. A distinct characteristic of these systems is the fact that the nodes can harvest energy 
throughout the duration in which communication takes place. As such, transmission policies of the 
nodes need to adapt to these harvested energy arrivals. In this paper, we consider optimization of point- 
to-point data transmission with an energy harvesting transmitter which has a limited battery capacity, 
communicating in a wireless fading channel. We consider two objectives: maximizing the throughput 
by a deadline, and minimizing the transmission completion time of the communication session. We 
optimize these objectives by controlling the time sequence of transmit powers subject to energy storage 
capacity and causality constraints. We, first, study optimal offline policies. We introduce a directional 
water-filling algorithm which provides a simple and concise interpretation of the necessary optimality 
conditions. We show the optimality of an adaptive directional water-filling algorithm for the throughput 
maximization problem. We solve the transmission completion time minimization problem by utilizing 
its equivalence to its throughput maximization counterpart. Next, we consider online policies. We use 
stochastic dynamic programming to solve for the optimal online policy that maximizes the average 
number of bits delivered by a deadline under stochastic fading and energy arrival processes with causal 
channel state feedback. We also propose near-optimal policies with reduced complexity, and numerically 
study their performances along with the performances of the offline and online optimal policies under 
various different configurations. 
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I. Introduction 

This paper considers wireless communication using energy harvesting transmitters. In such a 
scenario, incremental energy is harvested by the transmitter during the course of data transmission 
from random energy sources. As such, energy becomes available for packet transmission at 
random times and in random amounts. In addition, the wireless communication channel fluctuates 
randomly due to fading. These together lead to a need for designing new transmission strategies 
that can best take advantage of and adapt to the random energy arrivals as well as channel 
variations in time. 

The simplest system model for which this setting leads to new design insights is a wireless link 
with a rechargeable transmitter, which we consider here. The incoming energy can be stored in 
the battery of the rechargeable transmitter for future use. However, this battery has finite storage 
capacity and the transmission policy needs to guarantee that there is sufficient battery space 
for each energy arrival, otherwise incoming energy cannot be saved and will be wasted. In this 
setting, we find optimal offline and online transmission schemes that adapt the instantaneous 
transmit power to the variations in the energy and fade levels. 

There has been recent research effort on understanding data transmission with an energy har- 
vesting transmitter that has a rechargeable battery [ID-El- In Q, data transmission with energy 
harvesting sensors is considered, and the optimal online policy for controlling admissions into 
the data buffer is derived using a dynamic programming framework. In [2], energy management 
policies which stabilize the data queue are proposed for single-user communication and under 
a linear approximation, some delay optimality properties are derived. In [3], the optimality 
of a variant of the back pressure algorithm using energy queues is shown. In [|4]|, throughput 
optimal energy allocation is studied for energy harvesting systems in a time constrained slotted 
setting. In Q, jU, minimization of the transmission completion time is considered in an energy 
harvesting system and the optimal solution is obtained using a geometric framework similar to 
the calculus approach presented in [8J. In [7], energy harvesting transmitters with batteries of 
finite energy storage capacity are considered and the problem of throughput maximization by a 
deadline is solved in a static channel. 

An earlier line of research considered the problem of energy management in communications 
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satellites J9]|, ifTOll . In ||9l, various energy allocation problems in solar powered communication 
satellites are solved using dynamic programming. In [fTOl , optimal energy allocation to a fixed 
number of time slots is derived under time-varying channel gains and with offline and online 
knowledge of the channel state at the transmitter. Another related line of research considered 
energy minimal transmission problems with deadline constraints (HI, lfTTI - lfT3l . Our work pro- 
vides optimal transmission policies to maximize the throughput and minimize the transmission 
completion time, under channel fluctuations and energy variations, in a continuous time model, 
generalizing these related works, [§]-[]8), [fT0l - [fl"3l , from various different perspectives. 

In particular, we consider two related optimization problems. The first problem is the maxi- 
mization of the number of bits (or throughput) transmitted by a deadline T. The second problem 
is the minimization of the time (or delay) by which the transmission of B bits is completed. 
We solve the first problem under deterministic (offline) and stochastic (online) settings, and we 
solve the second problem in the deterministic setting. We start the analysis by considering the 
first problem in a static channel under offline knowledge. The solution calls for a new algorithm, 
termed directional water-filling. Taking into account the causality constraints on the energy usage, 
i.e., the energy can be saved and used in the future, the algorithm allows energy flow only to the 
right. In the algorithmic implementation of the solution, we utilize right permeable taps at each 
energy arrival point. This solution serves as a building block for the fading case. Specifically, 
we show that a directional water-filling algorithm that adapts to both energy arrivals and channel 
fade levels is optimal. Next, we consider the second problem, i.e., the minimization of the time 
by which transmission of B bits is completed. We use the solution of the first problem to solve 
this second problem. This is accomplished by mapping the first problem to the second problem 
by means of the maximum departure curve. This completes the identification of the optimal 
offline policies in the fading channel. 

Next, we set out to find online policies. We address online scheduling for maximum through- 
put by the deadline T in a setting where fading level changes and energy arrives as random 
processes in time. Assuming statistical knowledge and causal information of the energy and 
fading variations, we solve for the optimal online power policy by using continuous time 
stochastic dynamic programming ffT3l . [[l"4l . To reduce the complexity required by the dynamic 
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programming solution, we propose simple online algorithms that perform near-optimal. Finally, 
we provide a thorough numerical study of the proposed algorithms under various system settings. 

II. System Model 

We consider a single-user fading channel with additive Gaussian noise and causal channel 
state information (CSI) feedback as shown in Fig. \T\ The transmitter has two queues, the data 
queue where data packets are stored, and an energy queue where the arriving (harvested) energy 
is stored. The energy queue, i.e., the battery, can store at most E max units of energy, which is 
used only for transmission, i.e., energy required for processing is not considered. 

The received signal y is given by y — \fhx + n, where h is the (squared) fading, x is the 
channel input, and n is a Gaussian random noise with zero-mean and unit- variance. Whenever 
an input signal x is transmitted with power p in an epoch of duration L, ~ log (1 + hp) bits of 
data is served out from the backlog with the cost of Lp units of energy depletion from the energy 
queue. This follows from the Gaussian channel capacity formula. The bandwidth is sufficiently 
wide so that L can take small values and we approximate the slotted system to a continuous 
time system. Hence, we say that if at time t the transmit power of the signal is x 2 (t) = p(t), 
the instantaneous rate r(t) in bits per channel use is 

r(t) = Uog (I + h(t)p(t)) (1) 

Following a model similar to |fl2). we assume that the fading level h and energy arrivals 
are stochastic processes in time that are marked by Poisson counting processes with rates A^ 
and A e , respectively. Therefore, changes in fading level and energy arrivals occur in countable 
time instants, which are indexed respectively as t{, t 2 , ■ ■ ■ , . . . and t\ , t|, . . . , t^, . . . with the 
convention that t\ — t\ — 0. By the Poisson property, the inter-occurrence times t{ — t{_ x and 
tj — t £ j-i are exponentially distributed with means 1/A/ and 1/A e , respectively. The fading level 
in [0, t{) is hi, in \t[,t{) is h 2 , and so on. Similarly, Ei units of energy arrives at time t\, and 
i?o units of energy is available at time 0. Hence {(t 4 e , Ei)}°l and {{t{ , hi)}^ completely define 
the events that take place during the course of data transmission. This model is shown in Fig. [2] 
The incoming energy is first buffered in the battery before it is used in data transmission, and 
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Fig. 1. Additive Gaussian fading channel with an energy harvesting transmitter and causal channel state information (CSI) 
feedback. 



the transmitter is allowed to use the battery energy only. Accordingly, we assume E^ < E max 
for all i as otherwise excess energy cannot be accommodated in the battery anyway. 

In the sequel, we will refer to a change in the channel fading level or in the energy level as an 
event and the time interval between two consecutive events as an epoch. More precisely, epoch 
i is defined as the time interval where U and U + i are the times at which successive 

events happen and the length of the epoch is Lj = t i+ i —ti. Naturally, energy arrival information 
is causally available to the transmitter. Moreover, by virtue of the causal feedback link, perfect 
information of the channel fade level is available to the transmitter. Therefore, at time t all {E { } 
and {hj} such that t\ <t and t J < t are known perfectly by the transmitter. 

A power management policy is denoted as p(t) for t E [0,T]. There are two constraints on 
pit), due to energy arrivals at random times and also due to finite battery storage capacity. Since 
energy that has not arrived yet cannot be used at the current time, there is a causality constraint 
on the power management policy as: 

/ p{u)du<^E h Vz (2) 
Jo j=0 

where the limit of the integral t\ should be interpreted as — e, for small enough e. Moreover, 
due to the finite battery storage capacity, we need to make sure that energy level in the battery 
never exceeds E max . Since energy arrives at certain time points, it is sufficient to ensure that 
the energy level in the battery never exceeds E max at the times of energy arrivals. Let d(t) = 
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Fig. 2. The system model and epochs under channel fading. 

max{tf : tf < t}. Then, 

m r t 

V Ej - / p(u)du < E max , Vt E [0, T] 
j= o J° 

We emphasize that our system model is continuous rather than slotted. In slotted models, e.g., 
0, 01, 0, the energy input-output relationship is written for an entire slot. Such models allow 
energies larger than E max to enter the battery and be used for transmission in a given single 
slot. Our continuous system model prohibits such occurrences. 

Our ultimate goal is to develop an online algorithm that determines the transmit power as a 
function of time using the causal knowledge of the system, e.g., the instantaneous energy state 
and fading CSI. We will start our development by considering the optimal offline policy. 

III. Maximizing Throughput in a Static Channel 

In this section, we consider maximizing the number of bits delivered by a deadline T, in a non- 
fading channel with offline knowledge of energy arrivals which occur at times {ti, t 2 , ■ ■ ■ , tjv} m 
amounts {Ei, E 2 , . . . , E^}. The epoch lengths are Li = U — ti-\ for i = 1, . . . , N with to = 0, 
and L N+ i = T — t N . There are a total of N + 1 epochs. The optimization is subject to causality 
constraints on the harvested energy, and the finite storage constraint on the rechargeable battery. 
This problem was solved in flT) using a geometric framework. Here, we provide the formulation 
for completeness and provide an alternative algorithmic solution which will serve as the building 
block for the solution for the fading channel presented in the next section. 

First, we note that the transmit power must be kept constant in each epoch [Hl-[|7l, due to the 
concavity of rate in power. Let us denote the power in epoch i by The causality constraints 
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in © reduce to the following constraints on pi, 



e-i 



J2 L iPi<J2 Ei > t=l,---,N + l (4) 

i=l i=0 

Moreover, since the energy level in the battery is the highest at instants when energy arrives, 
the battery capacity constraints in © reduce to a countable number of constraints, as follows 

i i 

^Ei-^LiPi^Erw, £=1,...,N (5) 

i=0 i=l 

Note that since E > 0, there is no incentive to make pi = for any i. Hence, pi > is necessary 
for optimality. 

The optimization problem is: 

N+l 

y]v i °s( i +Pi) ( 6 ) 

Pi>0 * — ' I 
i=l 

e t-\ 

s.t. J^LiPi^^Ei, £=1,...,N+1 (7) 

i=l i=0 

e i 

^Ei-^Lffli^E^, £=1,...,N (8) 

i=0 i=l 

We note that the constraint in © must be satisfied with equality for £ — N + 1, otherwise, we 
can always increase some pi without conflicting any other constraints, increasing the resulting 
number of bits transmitted. 

Note that the objective function in © is concave in the vector of powers since it is a sum 
of log functions, which are concave themselves. In addition, the constraint set is convex as it is 
composed of linear constraints. Hence, the above optimization problem is a convex optimization 
problem, and has a unique maximizer. We define the following Lagrangian function [15] for any 
Aj > and //j > 0, 

N+l J N+l I j j-1 \ V / j j 

£ = y log + pi "> ~ x j [ LtPi ~Y Ei ) [Y Ei ~Y LiPi ~ Ema - 

i=l j=l \i=l i=0 ) j=l \i=0 i=l 

(9) 

Lagrange multipliers {A/} are associated with constraints in © and {/Xj} are associated with 
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([8]). Additional complimentary slackness conditions are as follows, 



A, \Y, L iPi-Y, E A =°> 3 = h---,N (10) 

\i=l i=0 J 

^\^E l -Y J L i p l -E max \=Q, j = 1 iV (11) 

\i=0 i=l / 

In (flOl) . j = iV + 1 is not included since this constraint is in fact satisfied with equality, because 
otherwise the objective function can be increased by increasing some p^. Note also that as p { > 0, 
we did not include any slackness conditions for pj. 

We apply the KKT optimality conditions to this Lagrangian to obtain the optimal power levels 
p* in terms of the Lagrange multipliers as, 

Pi = 7 N+1 1 £ V" 1 ' «' = 1 >---,tf d 2 ) 

(E^ \ E, ,/',) 

and p^ +1 = — 1. Note that p* that satisfy Y^i^ ^*Pf = Ei=o-^> * s unique- 
Based on the expression for p* in terms of the Lagrange multipliers in (flZl . we have the 
following observation on the structure of the optimal power allocation scheme. 

Theorem 1 When E max = oo, the optimal power levels is a monotonically increasing sequence: 
P* +1 > p\- Moreover, if for some I, L iP*i < Eto ^' tnen P? = Pi+v 

Proof: Since E max = oo, constraints in ([8]) are satisfied without equality and /i« = for all i by 
slackness conditions in (fTTT) . From (fT2l) . since Aj > 0, optimum p* are monotonically increasing: 
> Pi - Moreover, if for some t, Yl\=\ LiP* < YliZo Ei, then \ e = 0, which means p\ = p} +1 . 

m 

The monotonicity in Theorem Q] is a result of the fact that energy may be spread from the 
current time to the future for optimal operation. Whenever a constraint in © is not satisfied 
with equality, it means that some energy is available for use but is not used in the current epoch 
and is transferred to future epochs. Hence, the optimal power allocation is such that, if some 
energy is transferred to future epochs, then the power level must remain the same. However, if 
the optimal power level changes from epoch i to % + 1, then this change should be in the form 
of an increase and no energy is transferred for future use. That is, the corresponding constraint 



in CO) is satisfied with equality. 

If E max is finite, then its effect on the optimal power allocation is observed through ^ in 
(IT2]) . In particular, if the constraints in dH) are satisfied without equality, then optimal p* are 
still monotonically increasing since //j = 0. However, as Ei < E max for all i, the constraint 
with the same index in © is satisfied without equality whenever a constraint in dHJ is satisfied 
with equality. Therefore, a non-zero /i; and a zero Aj appear in p* in (fl2l) . This implies that the 
monotonicity of p* may no longer hold. E max constraint restricts power levels to take the same 
value in adjacent epochs as it constrains the energy that can be transferred from current epoch 
to the future epochs. Indeed, from constraints in dH), the energy that can be transferred from 
current, say the ith, or previous epochs, to future epochs is E max — E^. Hence, the power levels 
are equalized only to the extent that E max constraint allows. 

A. Directional Water-Filling Algorithm 

We interpret the observed properties of the optimal power allocation scheme as a directional 
water-filling scheme. We note that if E units of water (energy) is filled into a rectangle of bottom 
size L, then the water level is -|. Another key ingredient of the directional water- filling algorithm 
is the concept of a right permeable tap, which permits transfer of water (energy) only from left 
to right. 

Consider the two epoch system. Assume E max is sufficiently large. If > j^, then some 
energy is transferred from epoch 1 to epoch 2 so that the levels are equalized. This is shown in 
the top figure in Fig. |3] However, if ^ < j^, no energy can flow from right to left. This is due 
to the causality of energy usage, i.e., energy cannot be used before it is harvested. Therefore, 
as shown in the middle figure in Fig. |3] the water levels are not equalized. We implement this 
using right permeable taps, which let water (energy) flow only from left to right. 

We note that the finite E max case can be incorporated into the energy-water analogy as a 
constraint on the amount of energy that can be transferred from the past to the future. If the 
equalizing water level requires more than E max — Ei amount of energy to be transferred, then 
only E max — Ei can be transferred. Because, otherwise, the energy level in the next epoch exceeds 
E ma x causing overflow of energy, which results in inefficiencies. More specifically, when the 
right permeable tap in between the two epochs of the example in bottom figure in Fig. |3] is 
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Fig. 3. Directional water-filling with right permeable taps in a two-epoch setting. 



turned on, only E max — E\ amount of energy transfer is allowed from epoch 1 to epoch 2. 

IV. Maximizing Throughput in a Fading Channel 

We now solve for the offline policy for the fading channel utilizing the insights obtained in the 
previous section. The channel state changes M times and energy arrives N times in the duration 
[0, T). Hence, we have M + N + 1 epochs. Our goal is again to maximize the number of bits 
transmitted by the deadline T. Similar to the non-fading case, the optimal power management 
strategy is such that the transmit power is constant in each event epoch. Therefore, let us again 
denote the transmit power in epoch % by for i = 1, . . . , M + N + 1. We define E in (i) as the 
energy which arrives in epoch i. Hence, E in {i) = Ej for some j if event i is an energy arrival 
and E in {i) = if event i is a fade level change. Also, E in {l) = E . Similar to the non-fading 
case, we have causality constraints due to energy arrivals and an E max constraint due to finite 
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battery size. Hence, the optimization problem in this fading case becomes: 

M+N+1 j 

max -77 log (1 + KVi) (13) 

Pi>0 *■ — ' i 
i=i 

t i 

s.t. ^2La>i<^2E in (i), W (14) 

i=i i=i 

E in (l) - L ^ ^ E ^ax , W (15) 

i=l i=l 

Note that, as in the non-fading case, the constraint in ([141 for £ = M + N + 1 must be satisfied 
with equality, since otherwise, we can always increase one of p. t to increase the throughput. 

As in the non-fading case, the objective function in (fT3l) is concave and the constraints are 
convex. The optimization problem has a unique optimal solution. We define the Lagrangian for 

any Aj, jii and r)i as, 



M+N+1 , M+N+1 / j j 

£= E ylog(l + E aW^L^-E^ 

i=l j=l \»=1 i=l 



Z 



M +7V+1 / j j \ M+iV+1 

~~ ^ A*i I ^ ^m(0 - ^2 L iPi - j + ^ (16) 

i=l \i=l t=l / i=l 

Note that we have not employed the Lagrange multipliers {r/j} in the non-fading case, since 
in that case, we need to have all > 0. However, in the fading case, some of the optimal 
powers can be zero depending on the channel fading state. Associated complimentary slackness 
conditions are, 

i j 



x 3 J2 L ^ - E = °' ^ (17) 

\i=l i=l J 

Hj ( ^ E in (i) - E LiPi - E max J = 0, Vj (18) 
.i=i i=i J 

rjjPj = 0, Vj (19) 



It follows that the optimal powers are given by 
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hi 



(20) 
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where the water level in epoch i, ui, is 



We have the following observation for the fading case. 

Theorem 2 When E max = oo, for any epoch i, the optimum water level Vi is monotonically 
increasing: u i+ \ > z/j. Moreover, if some energy is transferred from epoch i to i + 1, then 
Vi = v i+1 . 

Proof: E max = oo assumption results in /i, = for all z. From (1211 . and since A; > 0, we have 
^i+i > i^i- If energy is transferred from the zth epoch to the i + 1st epoch, then the ith constraint 
in (fl4l) is satisfied without equality. This implies, by slackness conditions in (fT7l) . that for those 
i, we have Aj = 0. Hence, by (I2TT) . z/j = u i+1 . In particular, i/j = za, for all epochs i and j that 
are in between two consecutive energy arrivals as there is no wall between these epochs and 
injected energy freely spreads into these epochs. ■ 

As in the non-fading case, the effect of finite E max is observed via the Lagrange multipliers /Xj. 
In particular, whenever E max constraint is satisfied with equality, the monotonicity of the water 
level no longer holds. E max constrains the amount of energy that can be transferred from one 
epoch to the next. Specifically, the transferred energy cannot be larger than E max — E in {i). Note 
that this constraint is trivially satisfied for those epochs with E in {i) = because E in {i) < E max 
and hence the water level in between two energy arrivals must be equalized. However, the next 
water level may be higher or lower depending on the new arriving energy amount. 

A. Directional Water-Filling Algorithm 

The directional water-filling algorithm in the fading channel requires walls at the points of 
energy arrival, with right permeable water taps in each wall which allows at most E max amount 
of water to flow. No walls are required to separate the epochs due to changes in the fading level. 
The water levels when each right permeable tap is turned on will be found by the directional 
water-filling algorithm. Optimal power allocation p* is then calculated by plugging the resulting 
water levels into (|20l> . An example run of the algorithm is shown in Fig. HI for a case of 12 
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Fig. 4. Directional water-filling with right permeable taps in a fading channel. 

epochs. Three energy arrivals occur during the course of the transmission, in addition to the 
energy available at time t = 0. We observe that the energy level equalizes in epochs 2, 4, 5, 
while no power is transmitted in epochs 1 and 3, since the channel gains in these epochs are 
too low (i.e., ^7 too high). The energy arriving at the beginning of epoch 6 cannot flow left due 
to causality constraints, which are enforced by right permeable taps, which allow energy flow 
only to the right. We observe that the energy equalizes between epochs 8 through 12, however, 
the excess energy in epochs 6 and 7 cannot flow right, due to the E max constraint enforced by 
the right permeable tap between epochs 7 and 8. 

V. Transmission Completion Time Minimization in Fading Channel 

In contrast to the infinite backlog assumption of the previous sections, we now assume that the 
transmitter has B bits to be communicated to the receiver in the energy harvesting and fading 
channel setting. Our objective now is to minimize the time necessary to transmit these B bits. 
This problem is called the transmission completion time minimization problem. In |0, [0, this 
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problem is formulated and solved for an energy harvesting system in a non-fading environment. 
In 0, the problem is solved when there is an E max constraint on the energy buffer (battery) 
by identifying its connection to its throughput optimization counterpart. Here, our goal is to 
address this problem in a fading channel, by using the directional water-filling approach we 
have developed so far. 

The transmission completion time minimization problem can be stated as, 

min T (22) 
- L 

s.t. Y,Y log( - 1 + hiPi ^ = B (23) 

1=1 

J2 L iPi<J2 E ™&> t = l,...,N (24) 

i=l i=l 

I I 

Ein(%) ~ L iPi < E max, I = h ■ ■ ■ , N (25) 

i=l i=l 

where N = N(T) is the number of epochs in the interval [0, T]. The solution will be a 
generalization of the results in [HI-ED for the fading case. To this end, we introduce the 
maximum departure curve. This maximum departure curve function will map the transmission 
completion time minimization problem of this section to the throughput maximization problem 
of the previous sections. 

A. Maximum Departure Curve 

Given a deadline T, define the maximum departure curve D{T) for a given sequence of energy 
arrivals and channel fading states as, 

JV(T) 

D(T) = max ^ y log (1 + h iPi ) (26) 

i=l 

where N(T) is the number of epochs in the interval [0, T\. The maximization in (l26l) is subject 
to the energy causality and maximum battery storage capacity constraints in (l24l and (1251) . The 
maximum departure function D(T) represents the maximum number of bits that can be served 
out of the backlog by the deadline T given the energy arrival and fading sequences. This is 
exactly the solution of the problem studied in the previous sections. Some characteristics of the 
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maximum departure curve are stated in the following lemma. 

Lemma 1 The maximum departure curve D(T) is a monotonically increasing and continuous 
function of T. D(T) is not differentiable at {if} and {t{}. 

Proof: The monotonicity follows because as the deadline is increased, we can transmit at least 
as many bits as we could with the smaller deadline. The continuity follows by observing that, if 
no new energy arrives or fading state changes, there is no reason to have a discontinuity. When 
new energy arrives, since the number of bits that can be transmitted with a finite amount of 
energy is finite, the number of bits transmitted will not have any jumps. Similarly, if the fading 
level changes, due to the continuity of the log function, D(T) will be continuous. 

For the non-differentiable points, assume that at t = if, an energy in the amount of E; t arrives. 
There exists a small enough increment from if that the water level on the right is higher than 
the water level on the left. The right permeable taps will not allow this water to flow to left. 
Then, the D(T) will have the following form: 

D(tt + A) = D(tt) + | log (l + ^ (27) 

Thus, the right derivative of D{T) at i = if, becomes arbitrarily large. Hence, D(T) is not 
differentiable at if. At i = if , the fade level changes from hi to h i+ i. As t is increased, water 
level decreases unless new energy arrives. The change in the water level is proportional to ^— j- 
for i > if and is proportional to -^ for t < if . Hence, at t = if , D(T) is not differentiable. ■ 

The continuity and monotonicity of D{T) implies that the inverse function of D(T) exists, 
and that for a closed interval [a, b], .D _1 ([a, b]) is also a closed interval. Since D{T) is obtained 
by the directional water- filling algorithm, the derivative of D(T) has the interpretation of the 
rate of energy transfer from past into the future at time T, i.e., it is the measure of the tendency 
of water to flow right. The non-differentiabilities at energy arrival and fading change points are 
compatible with this interpretation. 

We can visualize the result of Lemma \T\ by considering a few simple examples. As the 
simplest example, consider the non-fading channel (h = 1) with E units of energy available 
at the transmitter (i.e., no energy arrivals). Then, the optimal transmission scheme is a constant 
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transmit power scheme, and hence, we have, 

I>(T) = | tog (l + |?) (28) 

It is clear that this is a continuous, monotonically increasing function, whose derivative at T = 
(at the time of energy arrival) is unbounded. 

Next, we consider a two epoch case where E\ arrives at T x and fading level is constant (and 
also h = 1). We assume E and E\ are both smaller than E max and E + E\ > E max . After 
some algebra, D(T) can shown to be expressed as, 

' |log(l + f), 0<t<T 1 
flog (l + f ) + ^flog (l + A) , T x < t < T, 



D(t) 



(29) 

| log (1 + Mi), T 2 <t<T 3 



k f log (l + + ^ log (l + f^) , T 3 < t < 



OO 



where T 2 = ^ +T l5 T 3 = ^f^f^ ■ In this £ max constrained case, the asymptote of D(T) 
as T — )• oo is strictly smaller than that in S ma3; = oo case. 

In the most general case where we have multiple energy arrivals and channel state changes, 
these basic properties will follow. An example case is shown in Fig. [5J Note that there may be 
discontinuities in D'(T) due to other reasons than fading level changes and energy arrivals, such 
as the E max constraint. 

B. Solution of the Transmission Completion Time Minimization Problem in a Fading Channel 

We now solve the transmission completion time minimization problem stated in (l22l-(|25l). 
Minimization of the time to complete the transmission of B bits available at the transmitter is 
closely related with the maximization of the number of bits that can be sent by a deadline. In 
fact, if the maximum number of bits that can be sent by T is less than B, then it is not possible to 
complete the transmission of B bits by T. As we state formally below, if T* is the minimal time 
to complete the transmission of B bits, then necessarily B = D(T*). This argument provides 
a characterization for T* in terms of the maximum departure curve, as stated in the following 
theorem. 
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Eq Ei E-2 E3 



Fig. 5. The general form of the maximum departure curve. 

Theorem 3 The minimum transmission completion time T* to transmit B bits is T* = mm{t G 
M B } where M B = {t : B = D(t)}. 

Proof: For t such that D{t) < B, T* > t since the maximum number of bits that can be served 
by t is D(t) and it is less than B. Hence, B bits cannot be completed by t. Conversely, for 
t such that D(t) > B, T* < t because B bits can be completed by t. Hence, D(T*) = B is 
a necessary condition. As D(T) is continuous, the set {t : B = D(t)} is a closed set. Hence, 
min{£ : B = D(t)} exists and is unique. By the definition of T*, we have T* = min{t : B = 
D(t)}. * 

VI. Online Transmission Policies 

In this section, we will study scheduling in the given setting with online, i.e., causal, infor- 
mation of the events. In particular, we consider the maximization of the number of bits sent by 
deadline T given only causal information of the energy arrivals and channel fade levels at the 
transmitter side as in Fig. [T] 

We assume that the energy arrival is a compound Poisson process with a density function f e . 
Hence, N e is a Poisson random variable with mean \ e T. The channel fade level is a stochastic 
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process marked with a Poisson process of rate A/. Thus, Nf is Poisson with mean XfT. The 
channel takes independent values with probability density fh at each marked time and remains 
constant in between two marked points. 



A. Optimal Online Policy 

The states of the system are fade level h and battery energy e. An online policy is denoted as 
g(e, h, t) which denotes the transmit power decided by the transmitter at time t given the states e 
and h. We call a policy admissible if g is nonnegative, g(0, h, t) — for all h and t £ [0, T] and 
e(T) = 0. That is, we impose an infinite cost if the remaining energy in the battery is non-zero 
after the deadline. Hence, admissible policies guarantee that no transmission can occur if the 
battery energy is zero and energy left in the battery at the time of the deadline is zero so that 
resources are used fully by the deadline. The throughput J g (e,h,t) is the expected number of 
bits sent by the time t under the policy g 



r f* 1 

j -\og{l + h{r)g{e,h,r))dr 



J g (e, h,t) = E 

Then, the value function is the supremum over all admissible policies g 



(30) 



J(e, h, t) = sup J g 



(3D 



Therefore, the optimal online policy g*(e,h,t) is such that J(e,h,t = 0) = J g *, i.e., it solves 
the following problem 



max E 

9 



log (1 + h(r)g(e, h,r)) dr 



(32) 



In order to solve (l32l . we first consider 5- skeleton of the random processes |[T3l . For sufficiently 
small 5, we quantize the time by 5 and have the following. 



m&xE 

9 



T 



-log(l + h(r)g(e, h,r))dr 



5 



max - log (1 + h(0)g(e, h, 0)) + J(e - 6g(e, h, 0), h, 5) 

g(e,h,0) V 2 



(33) 



Then, we can recursively solve (1331 to obtain g*(e,h,t = k5) for k = 1,2,..., |_?J. This 
procedure is the dynamic programming solution for continuous time and the outcome is the 
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optimal online policy lfT3l . Ifl4l . After solving for g*(e, h, t), the transmitter records this function 
as a look-up table and at each time t, it receives feedback h(t), senses the battery energy e(t) 
and transmits with power g*(e(t), h(t),t). 

B. Other Online Policies 

Due to the curse of dimensionality inherent in the dynamic programming solution, it is natural 
to forgo performance in lieu of less complex online policies. In this subsection, we propose 
several suboptimal transmission policies that can somewhat mimic the offline optimal algorithms 
while being computationally simpler and requiring less statistical knowledge. In particular, we 
resort to event-based online policies which react to a change in fading level or an energy arrival. 
Whenever an event is detected, the online policy decides on a new power level. Note that the 
transmission is subject to availability of energy and the E max constraint. 

1 ) Constant Water Level Policy: The constant water level policy makes online decisions for 
the transmit power whenever a change in fading level is observed through the causal feedback. 
Assuming that the knowledge of the average recharge rate P is available to the transmitter and 
that fading density f h is known, the policy calculates ho that solves the following equation. 



Whenever a change in the fading level occurs, the policy decides on the following power level 
Pi = — j^j . If the energy in the battery is nonzero, transmission with pi is allowed, 
otherwise the transmitter becomes silent. 

Note that this power control policy is the same as the capacity achieving power control policy 
in a stationary fading channel lfT6l with an average power constraint equal to the average recharge 
rate. In Q, this policy is proved to be stability optimal in the sense that all data queues with 
stabilizable arrival rates can be stabilized by policies in this form where the power budget is 
P — e for some e > sufficiently small. However, for the time constrained setting, this policy is 
strictly suboptimal as will be verified in the numerical results section. This policy requires the 
transmitter to know the mean value of the energy arrival process and the full statistics of the 
channel fading. A channel state information (CSI) feedback is required from the receiver to the 
transmitter at the times of events only. 




(34) 
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2) Energy Adaptive Water-Filling: Another reduced complexity event-based policy is obtained 
by adapting the water level to the energy level in each event. Again the fading statistics is assumed 
to be known. Whenever an event occurs, the policy determines a new power level. In particular, 
the cutoff fade level h is calculated at each energy arrival time as the solution of the following 
equation 



1 1 



h \h h 



f(h)dh = E current (35) 



where E current is the energy level at the time of the event. Then, the transmission power level 
is determined similarly as p^ = — ~J . This policy requires transmitter to know the fading 
statistics. A CSI feedback is required from the receiver to the transmitter at the times of changes 
in the channel state. 

3) Time-Energy Adaptive Water-Filling: A variant of the energy adaptive water filling policy 
is obtained by adapting the power to the energy level and the remaining time to the deadline. 
The cutoff fade level h is calculated at each energy arrival time as the solution of the following 
equation. 

1 i - - - - - - ^current 



. f(h)dh = ^= (36) 



Then, the transmission power level is determined as pi = — i 



+ 



VII. Numerical Results 

We consider a fading additive Gaussian channel with bandwidth W where the instantaneous 
rate is 

r(t) = W\og(l + h(t)p(t)) (37) 

h(t) is the channel SNR, i.e., the actual channel gain divided by the noise power spectral density 
multiplied by the bandwidth, and p(t) is the transmit power at time t. Bandwidth is chosen as 
W = 1 MHz for the simulations. 

We will examine the deadline constrained throughput performances of the optimal offline 
policy, optimal online policy, and other proposed sub-optimal online policies. In particular, we 
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compare the optimal performance with the proposed sub-optimal online policies which are based 
on waterfilling [[Toll . The proposed sub-optimal online policies use the fading distribution, and 
react only to the new energy arrivals and fading level changes. These event -based algorithms 
require less feedback and less computation, however, the fact that they react only to the changes 
in the fading level and new energy arrivals is a shortcoming of these policies. Since the system is 
deadline constrained, the policies need to take the remaining time into account yet the proposed 
policies do not do this optimally. We will simulate these policies under various different settings 
and we will observe that the proposed sub-optimal policies may perform very well in some cases 
while not as well in some others. 

We perform all simulations for 1000 randomly generated realizations of the channel fade 
pattern and 5 = 0.001 is taken for the calculation of the optimal online policy. The rates of 
Poisson mark processes for energy arrival and channel fading A e and A/ are assumed to be 1. 
The unit of A e is J/sec and that of Xf is 1/sec. Hence, the mean value of the density function 
f e is also the average recharge rate and the mean value of fh is the average fading level. The 
changes in the fading level occur relatively slowly with respect to the symbol duration. 

f e is set as a non-negative uniform random variable with mean P, and as the energy arrival 
is assumed to be smaller than E max , we have 2P < E max . Selection of the E max constraint is 
just for illustration. In real life, sensors may have batteries of E max on the order of kJ but the 
battery feeds all circuits in the system. Here, we assume a fictitious battery that carries energy 
for only communication purposes. Hence, E max on the order of 1 J will be considered. We will 
examine different fading distributions fh. In particular, Nakagami distribution with different shape 
parameter m will be considered. We implement the specified fading by sampling its probability 
density function with sufficiently large number of points. 

In order to assess the performance, we find an upper bound on the performances of the policies 
by first assuming that the channel fading levels and energy arrivals in the [0, T] interval are known 
non-causally, and that the total energy that will arrive in [0, T] is available at the transmitter at 
time t — 0. Then, for the water level p w that is obtained by spreading the total energy to the 
interval [0,T], with the corresponding fading levels, yield the throughput T ub defined in the 
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following 

T ub = jr 1 \ lo S + h i (p» " ^) + j (38) 

as an upper bound for the average throughput in the [0, T] interval; here k denotes the duration of 
the fade level in the ith epoch. Even the offline optimal policy has a smaller average throughput 
than T ub as the causality constraint does not allow energies to be spread evenly into the entire 
interval. 

We start with examining the average throughput of the system under Rayleigh fading with 
SNR= dB and deadline T = 10 sec, E max = 10 J as depicted in Fig. [6J We observe that 
time-energy adaptive waterfilling policy performs quite close to the optimal online policy in the 
low recharge rate regime. It can be a viable policy to spread the incoming energy when the 
recharge rate is low; however, its performance saturates as the recharge rate is increased. In this 
case the incoming energy cannot be easily accommodated and more and more energy is lost due 
to overflows. Similar trends can be found in Fig. [7] under very low recharge rate regime in the 
same setting with only difference being the battery capacity E max = 1 J. Next, we examine the 
setting with T = 10 sec, E max = 10 J under Nakagami fading of m = 3 (average SNR= 5 dB) 
and we observe similar performances as in the previous cases. As a common behavior in these 
settings, energy adaptive water-filling performs poorer with respect to the constant water level 
and time-energy adaptive water-filling schemes. 

Finally, we examine the policies under different deadline constraints and present the plots 
for Nakagami fading distribution with m . = 5 in Fig. [9j A remarkable result is that as the 
deadline is increased, stability optimal [2] constant water level policy approaches the optimal 
online policy. We conclude that the time-awareness of the optimal online policy has less and less 
importance as the deadline constraint becomes looser. We also observe that the throughput of the 
energy-adaptive waterfilling policy is roughly a constant regardless of the deadline. Moreover, 
the time-energy adaptive policy performs worse as T is increased because energies are spread 
to very long intervals rendering the transmit power very small and hence energy accumulates in 
the battery. This leads to significant energy overflows since the battery capacity is limited, and 
the performance degrades. 
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Fig. 6. Performances of the policies for various energy arrival rates under unit-mean Rayleigh fading, T = 10 sec and 

Emax — 10 J. 
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Fig. 7. Performances of the policies for various average recharge rates under unit-mean Rayleigh fading, T = 10 sec and 

E ma x — 1 J. 



VIII. Conclusions 

We developed optimal energy management schemes for energy harvesting systems operating in 
fading channels, with finite capacity rechargeable batteries. We considered two related problems 
under offline knowledge of the events: maximizing the number of bits sent by a deadline, and 
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Fig. 8. Performances of the policies for different energy recharge rates under Nakagami fading with m = 3, T — 10 sec and 

Emax — 10 J. 




Fig. 9. Performances of the policies with respect to deadline T under Nakagami fading distribution with m — 5 and average 
recharge rate P = 0.5 J/sec and E max = 10 J. 

minimizing the time it takes to send a given amount of data. We solved the first problem 
using a directional water-filling approach. We solved the second problem by mapping it to the 
first problem via the maximum departure curve function. Finally, we solved for throughput 
optimal policy for the deadline constrained setting under online knowledge of the events using 
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dynamic programming in continuous time. Our numerical results show the performances of these 
algorithms under offline and online knowledge. 



References 

[1] J. Lei, R. Yates, and L. Greenstein, "A generic model for optimizing single-hop transmission policy of replenishable 

sensors," IEEE Trans. Wireless Commun., vol. 8, pp. 547-551, February 2009. 
[2] V. Sharma, U. Mukherji, V. Joseph, and S. Gupta, "Optimal energy management policies for energy harvesting sensor 

nodes," IEEE Trans. Wireless Commun., vol. 9, pp. 1326-1336, April 2010. 
[3] M. Gatzianas, L. Georgiadis, and L. Tassiulas, "Control of wireless networks with rechargeable batteries," IEEE Trans. 

Wireless Commun., vol. 9, pp. 581-593, February 2010. 
[4] C. Ho and R. Zhang, "Optimal energy allocation for wireless communications powered by energy harvesters," in IEEE 

I SIT, June 2010. 

[5] J. Yang and S. Ulukus, "Transmission completion time minimization in an energy harvesting system," in CISS, March 
2010. 

[6] J. Yang and S. Ulukus, "Optimal packet scheduling in an energy harvesting communication system," IEEE Trans. Comm., 

submitted June 2010. Also available at larXiv: 1010. 1295 1. 
[7] K. Tutuncuoglu and A. Yener, "Optimum transmission policies for battery limited energy harvesting nodes," IEEE Trans. 

Wireless Comm., submitted, September 2010. Also available at [arXiv: 1010.6280]. 
[8] M. Zafer and E. Modiano, "A calculus approach to energy-efficient data transmission with quality-of-service constraints," 

IEEE/ACM Trans, on Networking, vol. 17, pp. 898-911, June 2009. 
[9] A. Fu, E. Modiano, and J. Tsitsiklis, "Optimal energy allocation and admission control for communications sattelites," 
IEEE/ACM Trans, on Networking, vol. 11, pp. 488-500, June 2003. 
[10] A. Fu, E. Modiano, and J. N. Tsitsiklis, "Optimal transmission scheduling over a fading channel with energy and deadline 

constraints," IEEE Trans. Wireless Commun., vol. 5, pp. 630-641, March 2006. 
[11] B. Prabhakar, E. Uysal-Biyikoglu, and A. El Gamal, "Energy-efficient transmission over a wireless link via lazy scheduling," 

in IEEE INFOCOM, April 2001. 
[12] E. Uysal-Biyikoglu and A. El Gamal, "On adaptive transmission for energy efficiency in wireless data networks," IEEE 

Trans. Inform. Theory, vol. 50, pp. 3081-3094, December 2004. 
[13] M. Zafer and E. Modiano, "Continuous time optimal rate control for delay constrained data transmission," in Allerton 

Conference, September 2005. 
[14] D. P. Bertsekas, Dynamic Programming and Optimal Control. Athena Scientific, 2007. 
[15] D. P. Bertsekas, Convex Analysis and Optimization. Athena Scientific, 2003. 

[16] A. Goldsmith and P. Varaiya, "Capacity of fading channels with channel side information," IEEE Trans. Inform. Theory, 
vol. 43, pp. 1986-1992, Nov. 1997. 



24 



